Video surveillance systems have wide application and can be used in a number of settings such as residential, commercial, governmental and industrial. Typically, one or more video cameras are strategically positioned in a protected or secure area and used to capture images of people and other objects that come within the range of the cameras. Video surveillance systems can transmit surveillance information across networks via wire-based and wireless technology, and across the Internet.
The feed from the video cameras in a surveillance system is usually directed to a monitoring station, such as a security guard station, or a central service, that provides monitoring services to a number of protected areas. A live video feed requires constant vigilant monitoring in order to detect security breach events. If there is no user who is actively watching the camera images, suspicious or undesired activity could go undetected.
Alternatively or in addition to real-time monitoring, many surveillance systems record and store surveillance footage to video tape or other medium for future reference and analysis. However, review of recorded footage alone may be insufficient in real-time applications that require immediate detection of security breaches.
In those systems in which video footage is recorded, all activity captured by a camera is typically saved, requiring a user to watch what are generally lengthy footages of video, much of it likely being uneventful and monotonous to the user, in order to locate the relevant sequences that provide information about security breach events. This review process requires diligent time commitment on the part of a monitoring user, and the delay associated with reviewing footage after it is recorded may result in loss of intervention opportunities.
Humans are limited in the amount of visual information they can process in tasks like video monitoring, whether live or recorded. After a time, significant security events can easily go unnoticed. In addition, in order to fully monitor an area, multiple cameras are generally required. As the number of cameras in a surveillance system increases, however, the amount of raw information that needs to be processed and analyzed also increases. Monitoring effectiveness is greatly reduced when multiple live videos must be monitored, and the amount of time needed to review stored footage likewise greatly increases with additional stored footage from multiple cameras.
Typically, the stored video recordings from a surveillance system are unstructured and unindexed. Most of the recorded footage is uninteresting and unimportant. Without an efficient means to locate significant security events in the footage, it is not cost-effective for security personnel to monitor or record the output from all available cameras in a multi-camera surveillance system.
A system that captures video images and/or audio without requiring a user to actively monitor the captured images or audio or requiring the user to later review voluminous amounts of recordings is needed.
In order to assist users monitoring video images from security cameras, some security or surveillance systems have incorporated motion detection sensors or functionality. Motion detection sensors may include infrared sensors which detect changes in temperature in the monitored area, or ultrasonic/microwave sensors that detect moving objects in their field of detection. Alternatively, a method of analyzing video from surveillance cameras to automatically detect motion can be used. When motion is detected, whether by external sensor or video motion detection, a signal is delivered to a monitoring system, which may sound an alarm, trigger video recording, or report the incident to a security guard station or central service as a potential security breach.
However, in complex security systems with a large number of cameras covering an area with a wide variety of possible activities, simple motion detection, whether through external sensors or video detection, is inadequate. That is, if every motion detected by the system caused the system to generate an alert, many “false alarms” would be created.
It is desirable to provide an improved surveillance system that reduces the number of false alarms by performing analysis on multiple video feeds to intelligently detect significant events.
In addition, with current surveillance systems, a receiver of an alarm sent by the surveillance system only knows that an event which may or may not be a security breach has occurred. The receiver has no other knowledge by which he can determine if the alarm signals the presence of a real intruder, or if it is a false alarm. In addition, the receiver has no other knowledge that reflects the level or type of potential security breach.
A system that automatically detects the occurrence of a significant event based upon data captured and transmitted from multiple cameras, optional detection equipment, and other stored information is needed. Upon determining that a significant event has occurred, the system should take action appropriate for the determined significant event, such as automatically generating and transmitting a notification in a manner that assists the receiver in understanding the nature of the event and taking action most appropriate for the determined event.
The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, it should not be assumed that any of the approaches described in this section qualify as prior art merely by virtue of their inclusion in this section.